In its simplest form, Pyramid's Natural Language Query (NLQ) engine takes a user's question and searches for the entities (hierarchies, members, and measures) in the current data model and applies basic analytic functions (like filtering and sorting) and builds a query against the data source. It then returns a smart AI-driven visualization comprising the results. NLQ can also be used to continue updating, changing, and even interacting with the query via the NLQ chat panel.
However, NLQ engine can extend these basic tenants by implementing several advanced intelligence capabilities related to time, geospatial and proper nouns.
NLQ Time Intelligence
The NLQ Time Intelligence brings an extra layer of AI driven querying to the user by allowing the user to write classic analytic date-time based questions in plain English. The engine can then use these as a basis for either finding the right date-time elements in the data source or for building formulations of time automatically to draw the appropriate result.
For time intelligence to operate, there must be date-time hierarchies in the model either by data type; by category type or by name.
Time Intelligence Cues
To use the time intelligence, you must ensure that the data source has date-time attributes; that these entities (hierarchies) that either appropriately named or categorized as date-time related. This can be done in the data modeling tools within Pyramid, or you can add an overlay to your view of the data model in the admin console or via Discover.
The following key concepts demonstrate how to employ the time intelligence logic:
- By period: by year, by quarter, or by month - both generically and specifically.
- Generics: "Show sales by promotion by month" will draw a set of line charts for sales by promotion by month (found by name or column type).
- Specifics: "Show sales by promotion in 2018" will draw a chart of sales by promotion for 2018 data. The 2018 data will be auto-calculated using calendar periods Jan 1 2018 through Dec 31 2018.
- Last period: last year, last month, last week,
- "Show sales by promotion for last year" will draw a chart of sales by promotion for 2019 data by using the current year's date (2020). The 2019 data will be auto-calculated using calendar periods Jan 1 2019 through Dec 31 2019.
- Last x periods: last 3 years, last 6 months, last 2 weeks
- "Show sales by promotion for last 2 years" will draw a chart of sales by promotion for 2018 through 2020 data by using the current date (2020). The 2 years of data will be auto-calculated using calendar periods Jan 1 2018 through Dec 31 2020. Note: In the event the data does not extend into that time span, it will be measured as of data ending the last date in the system.
- This grouped period: this year, this quarter, this month, this week
- "Show sales by promotion for this month" will draw a chart of sales by promotion for June 2020 using the current date (June 30th 2020). The month of data will be auto-calculated using calendar periods June 1 2020 through June 30 2020. Note: In the event the data does not extend into that time span, it will be measured as the of data ending the last date in the system.
- Expected in x periods: for example, for the next 2 weeks. This makes use of forecasting.
Non-SQL based Data Sources
NLQ does not support the same level of time intelligence operations for MS OLAP, Tabular and SAP BW data models, so it's not possible to specify a date or time period. This means that asking a question such as "Product sales last year" will not return a query. However, it is possible to group or classify a query by a date or time period. For example "Product sales by year" will work if there is a hierarchy or attribute in the model (or cube) called "year".
Note: It is possible to use NLQ forecasting with these models if they have the appropriate model structures.
NLQ Geo Intelligence
The NLQ Geo Intelligence brings an extra layer of AI driven querying to the user by allowing the user to write questions in plain English referencing global landmarks - notably countries, states and cities. The engine can then use these as a basis for either finding the right member elements in the data source to draw the appropriate result.
Note: For geo-intelligence to operate, there must be geospatial hierarchies in the model either by category type or by name.
Geo Intelligence Cues
To use the geo intelligence, you must ensure that the data source has geospatial attributes that are properly categorized as geospatial related. Further, the elements must be named in the standard way they would appear in English.
The following key concepts demonstrate how to employ the geo intelligence logic:
- By Country: both generically and specifically.
- Generics: "Show sales by country" will draw a map of sales all countries - as designated by the attribute type "country".
- Specifics: "Show sales by promotion in Australia" will draw the relevant chart for sales by promotion and filter the data using the country attribute, specifically on Australia.
- By State: both generically and specifically.
- Generics: "Show sales by state" will draw a map of sales all states - as designated by the attribute type "state".
- Specifics: "Show sales by promotion in New York" will draw the relevant chart for sales by promotion and filter the data using the state attribute, specifically on New York. "Show sales by promotion in New York and California" will include both states as a multi-select slicer to the report.
- By City:
- Specifics: "Show sales by promotion in Boston" will draw the relevant chart for sales by promotion and filter the data using the city attribute, specifically on Boston. Importantly, the city name needs to be capitalized.
NLQ Proper Noun Intelligence
Similar to the "Geo" intelligence, the NLQ Proper Noun Intelligence allows the user to write questions using the names of people. The engine can then use these as a basis for either finding the right member elements in the data source to draw the appropriate result.
Note: For proper noun-intelligence to operate, there must be categorized hierarchies of type "person".
Proper Noun Cues
- The use of capitalized text is required - indicating to the engine that the entity provided is a name. For example "Show sales by John last year"
- The a full name is provided, the name must be held in a single hierarchy. The engine will not match up to 2 separate hierarchies for first and last name.
- The names that will be recognized are those most popularly found in general use, in English.
Advanced Analytical Methods
Forecasting
You can perform forecasting via the NLQ by asking questions referring to future time periods. It's crucial to use correct grammar and prepositions when asking forecasting questions; use of the preposition "the" before the unit of time is particularly important, as it ensures correct interpretation of the date/ time unit. For example all 3 of these examples will attempt to forecast data out into the future: "Trend sales by month"; "Forecast expenses for the next 3 years"; "What will my costs be in the next 12 months"
Outliers
You can find statistical outliers in the current result set via the NLQ by asking to simply show them, This will generate the outlier flags as a dynamic hierarchy that can then be used for further analysis (slice, dice etc.). To show outliers simply write "Show sales by promotions with outliers". In the Chat bot, you can write "Add outliers"
Explain
You can use the Explain capability to generate a partial decision tree segment analysis of the specified data point, retrieving the most dominant drivers of the value or its key Influencers - depending on how the question is asked. To retrieve an explanation for a visible data point in the current visualization simple write "Explain why bike sales in the United States are so high". If the question is asked to "deconstruct", instead of explain (for influencers), the dominant drivers are returned.